Part 2
Teemu Säilynoja
2024-03-12
A: Continuous data
Most commonly KDE based density plots.
C: Discrete data
Fewer commonly used tools for effective visual checks.
B: Summary statistic
Another common visual PPC. Already more task specific.
D: Data split into groups
Use tools from A - C to assess predictions for subgroups of data.
Predictive checks present on many stages of Bayesian Workflows
Early stages of model building can be very exploratory
Clear guidelines reduce ad-hoc decisions during the exploration and assessment
\(\Rightarrow\) fewer mistakes
Aim: Provide structured recommendations on which visual predictive checks to use.
Aim: Provide structured recommendations on which visual predictive checks to use.
Emphasize discreteness of the predictive distribution
Our solution (bottom) returns the visualisation to a interval plot
Use pool-adjacent-violators (PAV) algorithm to replace binning in calibration plots with conditional event probabilities (CEP) (Dimitriadis, Gneiting, and Jordan (2021)).
Discrete outcomes make direct inspection of residuals difficult.
Binned residual plots are a common solution